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Introduction 


The  generalized  additive  channel  was  introduced  in  [1],  It  is  described 
by  an  additive  noise  process  with  sample  functions  inducing  a  measure  on  a 
linear  topological  vector  space,  and  by  a  constraint  which  includes 
dimensionality.  The  coding  capacity  of  the  matched  channel  was  analyzed  in 

with  an  exact  value  obtained  for  the  Gaussian  channel  and  an  upper  bound 
for  a  class  of  nonGaussian  channels.  Bounds  on  the  coding  capacity  for  the 
mismatched  Gaussian  generalized  channel  were  obtained  in  [2]. 

In  this  paper,  the  exact  coding  capacity  of  the  mismatched  Gaussian 
generalized  channel  is  determined,  along  with  an  upper  bound  for  a  class  of 
nonGaussian  mismatched  channels.  The  set  of  admissible  constraints  is  also 
greatly  increased  over  that  considered  in  [2].  Although  the  treatment  here  is 
restricted  to  noise  measures  induced  on  a  separable  Hilbert  space,  it  can 
readily  be  seen  that  the  results  extend  immediately  to  the  class  of  linear 
topological  vector  spaces  considered  in  E^]-  The  results  of  the  present  paper 
are  partly  based  on  the  Hilbert  space  results  on  information  capacity  given  in 
[3],  for  the  extension  to  linear  topological  vector  spaces,  one  would  use  the 
corresponding  results  given  in  [4].  The  focus  on  Hilbert  space  is  useful  for 

application  of  the  results  given  here  to  the  discrete-time  or  continuous-time 
additive  channel. 

The  basic  path  followed  here  is  well-known  to  information  theorists, 
appearing  in  the  analysis  of  much  simpler  channels.  A  generalization  of 
Fcinstein  s  Fundamental  Lemma  is  used  to  obtain  a  lower  bound  on  capacity,  and 
Fano  s  inequality  is  used  to  obtain  an  upper  bound.  However,  the  generality  of 
the  model  requires  a  development  considerably  different  from  that  of  the 
classical  treatment;  central  to  these  results  is  the  spectral  representation 
of  unbounded  self-adjoint  operators. 
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determining  bounds  on  coding  capacity  of  the  continuous- time  channel.  These 
bounds  will  be  given  elsewhere. 


Is  defined  as  the  zero  mean  Gaussian  cylinder  set  measure  on  H  having 


the  same  covariance  operator  as  The  entropy  Hgj^(N)  of  with  respect  to 
^GN  defined  as  follows.  Let  be  any  finite— dimensional  subspace  of  H, 
with  p^  and  p^j^  the  measures  induced  on  H  by  the  projection  operator 


Pjj  :  H  Let  Hqjj(N|H^)  be  the  entropy  of  p^  with  respect  to 

^GN^^^^n^  =  <»  if  it  is  false  that  p^  <<  while  otherwise 

r  dp?  1 

H  C  H,n>l  ^ 

n 


The  induced  measures  ^d  are  always  countably  additive  for  any  finite¬ 
dimensional  subspace  while  the  measure  ^gj^  will  be  countably  additive  if 


and  only  if  is  trace-class. 


Since  exists  and  range(R2^)  C  range(I^).  R^^  =  R|(I-hS)R|  for  a  self- 
adjoint  linear  operator  S.  with  (I+S)“^  existing  and  bounded  [5].  0  is  the 
smallest  limit  point  of  the  spectrum  of  S.  A  limit  point  of  the  spectrum  is 
either  the  limit  of  a  sequence  of  distinct  eigenvalues,  or  an  eigenvalue  of 
infinite  multiplicity,  or  a  point  of  the  continuous  spectrum  [6]. 


Coding  Capacity 


Theorem  1:  (1)  If  Hgjj(N)  <  “,  then 
C^(P)  <  t  log 
(2)  If  Hgjj(N)  <  «>  and  dim(H)  <  «>,  then  C^(P) 


=  0. 


(3)  If  is  Gaussian  and  dim(H)  =  «»,  then  C^(P)  =  z 


log  1 


-H 


P 

1+0 


Proof:  The  complete  theorem  will  first  be  proved  under  the  assumption  that 
0  <  «>. 
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Suppose  that  is  Gaussian,  with  0  <  <».  We  will  show  that 

CV(P)  >  4  -  lie 

Fix  any  6^0.  Since  1+0  is  the  smallest  limit  point  of  the  spectrum  of 
the  self-adjoint  operator  I  +  S,  there  exists  an  infinite  o.n.  set  {v  ,  n  >  1} 
in  the  range  of  the  projection  operator  where  {P^,  t  €  K}  is  the  left- 

continuous  resolution  of  the  identity  for  I  +  S  such  that  x  £  25(1+8)  if  and 
only  if  /qX  dllP^xll  <  “>,  and  then  (I+S)x  =  /^XdPj^x  where  the  integral  exists 
as  a  limit  in  the  strong  operator  topology  [6]. 

If  X  is  any  element  in  span{v  ,  n  >  1},  then  P  x  =  x  for  t  >  1+0+5,  since 

mm  M 

then  P  2  <x.v  >v  =  2  <x,v  >P  v  =  2  <x.v.>v..  Thus,  if  x  is  in 

^i=l  \  ^  i=l  ^  ^  ^  i=l  ^  ^ 


span{v^ , . . , v^} ,  then 


J“"t^d<PtX.x>  =  t^d<P^x.x>  =  t^dllP^xll^ 

<  (1+0+X)^  dllP^xll^  <  (l+0+X)^IIxll^. 


This  also  shows  that  span{v^.  n>l}  is  contained  in  2)(I+S),  and  that 
ll(I+S)xll^  <  (1+0+5)^11x11^  for  all  x  in  span{v^,  n>l}.  Similarly, 

ll(I+S)^xll  <  (1+0+5)11x11^  if  X  €  span{v^,  n>l}. 

Let  U  be  the  unitary  operator  in  H  which  satisfies  r|(I+S)2U^  =  r|  [5], 

For  each  v^,  define  u^  =  Uv^.  so  that  (I+S)2u\^  =  {1+S)K^. 

Choose  Q  in  (0,P).  For  n  >  1,  define  ji^  to  be  the  zero— mean  Gaussian 

O  ^  J-  d. 

measure  with  covariance  operator  2  R^Uj0R^Uj .  Let 

X  J.  J_ 

=  span{R^Uj^ ,  . .  .  ,R^u^} .  Note  that  C  raiige(R^),  because  = 

R^(I+S)^U  Uv^  =  since  =  1*  this  shows  that 

range (RJ)]  =  1.  Let  and  be  the  joint  cylinder  set  measures 
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defined  by  and  Since  gives  full  measure  to  H^,  we  can  replace  by 

the  measure  .  where  is  the  projection  operator  with  range  equal  to 

H  .  Thus  the  joint  measure  of  interest  is  concentrated  on  H  xH  ,  and  if  B,  and 

n  n  1 

are  Borel  sets  in  then  /^[BiXB2]  =  pj0fjj^{(x.y) :  (x.  x+P^y)  €  B^xB^}. 
Similarly.  (x.y) :  x+P^y  €  B2}.  Since  both  and  p^JSp^  are 

countably  additive  measures  on  the  results  of  [3]  can  be  applied.  Set 

=  {x  €  range  (R^):  llxll^  <  nP}. 

It  will  now  be  shown  that  1^[F^]  ^  0  as  n  ^  Note  that  =  U 

where  p^  is  the  zero-mean  Gaussian  measure  with  covariance  operator 

Q  n  n 

^  u.0u.,  so  that  x  =  2  <x.u  >u  a.e.  dp  (x) .  Thus 

i=l  i=l  ^  ^ 

^  n^^n^  ^  ^  n^^'  >  nP}  =  U  {x:  ll(I+S)2u\l|^  >  nP} 

XT  T^ 

=  P-  ll(I+S)^U^  2  <u.,x>u.ll^  >  nP} 

T^  i_l  1  1 

2 

<  p  ^{x:  (l+Gx-5)  2  <u..x>^  >  nP}. 
i=l  ^ 

The  random  variables  {^Uj,*>,  i^n}  are  i.i.d.  Gaussian  random  variables  with 
respect  to  p  mean  zero  and  variance  Q/[l+e4-5].  Applying  Chebyshev’s 

2 

inequality,  one  has  pJCF^]  <  — .  so  that  p^[f'^]  -»  0  as  n 

^  [nP-nQ]^  ^  ^ 

From  the  proof  of  Prop.  2  of  [7]. 


n 


t  2^(aj(x.y)  -  b^(x.y))  +  t  n  log(l  +  j^;^) 


where  {a^ ,  .  .  ,  .  ,b^}  is  a  family  of  i.i.d.  GaussiEin  random  variables 

with  respect  to  each  having  zero  mean  and  variance 


Q/(l+©+-6) 

2  _ 

Q  1 

[l+Q/(l+&t6) 

.i+e+6+Q. 

Take  nr  >  0,  and  define 
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-  V 

t 


=  t  n  log 


1  + 


i+e+5 


n 


-  nnr. 


An  =  {(x.y):  log 


“(x.y)  >  a  }, 


c  ^  2  z 

so  that  =  {(x,y);  ^  2  (a^  -  b^)  <  -nnr}.  Since  the  sequence  of  random 

i=l 

2  2 

variables  (a^  “  independent  and  have  zero  mean  w.r.t.  jj^.  Chebyshev’s 

i2 


inequality  gives  M^[An]  <  dn 


n  nr 


l+e+6+Q. 


Let  R  <  2  log 


1  + 


i+e+5 


0. 


nR. 


k  e 
n 


and  set  =  [e  ].  Then. 

■] 


^  ®  -By  the  Thomas ian-Kadota  gemeralization  of 

Feinstein’s  Fundamental  Lemma  (see.  e.g. .  [1.  p.  165]).  there  exists  a  code 

-a 

^^n’^n'^n^  with  <  k^e  +  f^CA^)  +  f^C^n^  •  From  above,  both  M^(A^)  and 


H  c  ^ 

M^CFj^]  tend  to  zero  as  n  Considering  k^e  choose  t  so  that 

0  1  ““ 

^  .  Then  k^e  ^  0  also. 

Q 


-a 


R  +  nr  <  t  log 


1  + 


i+a+5 


This  shows  that  any  rate  less  than  f  log 


1  + 


is  admissible,  for 


H^6j 

all  Q  <  P  and  for  all  6  >  0.  Hence,  the  supremum  over  all  admissible  rates 


1+0 


.  so  that  C^(P)  >  2  log 


1  + 


1+0 


when 


IS 


must  be  at  least  ^  log  1  + 

Gaussian. 

Now  consider  the  case  of  a  possibly  nonGaussian  not  necessarily 

countably  additive,  with  0  <  «>  and  Hgj^(N)  <  <».  Proceeding  exactly  as  in  the 

proof  of  this  result  for  the  matched  channel  [1.  pp.  167-168].  it  is  found 

that  any  admissible  R  must  satisfy  R  <  limsup  ^cJ(P).  C^(P)  is  the  informa- 

n 

tion  cap>a.city  of  the  additive  Gaussian  channel  with  noise  covari since  operator 
Rj^,  subject  to  the  constraints  that  support (ju^)  has  linear  dimension  <  n  and 
Xyllxll^d)u^(x)  <  nP. 
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lim  - =  i 


1  + 


1+0 


.  To  show 


It  now  remains  only  to  verify  that  lim  — - =  ^  log 

n  n 

this,  one  can  apply  Theorem  2  of  [[3] .  If  the  operator  S  has  no  eigenvalues 

nP 


less  than  0,  then  C^(nP)  =  log 

p 

exists  eind  equals  f  log  1  + 


1  + 


n[l+0] 


1+0 


for  all  n  >  1,  so  lim  ^  C^(nP) 

n 


If  the  operator  S  has  a  finite  set  of  eigenvalues  less  than  0,  <  X^  < 

...  <  Xj^  <  0,  then  +  nP  >  K0  for  sufficiently  large  n,  so  that  applying 

Theorem  2(c)  of  [3], 


K 


1=1 


and  this  again  converges  to  the  limit  t  log 


'1+0  ■ 

+  2  log 

nP  + 

1  _L 

1+X. 

1-' 

nP  +  2^0^ 

n(l+0) 


1  + 


1+0  I  • 


Initially,  suppose  that  S  has  an  infinite  sequence  of  eigenvalues  (X  ) 

'■  n'' 

strictly  less  than  0.  Since  0  is  the  smallest  limit  point  of  the  spectrum. 

K 

This  means  that  for  any  fixed  P,  KP  +  2  X.  >  KX„  for  all  sufficiently 

i=l  ^  ^ 


large  K.  To  see  this,  one  notes  that  for  any  A  >  0,  there  exists  Mq  such  that 
0  -  Xj  <  A  for  i  >  Mq.  Thus,  for  K  >  Mq, 


K  **0 

KX^  -  2  X.  i  I  (Xg-X  )  + 
1  =  1  1=1 


(K-Mq)A  < 


2  (0-X  )  +  (K-M  )A, 
i=l  ^  ^ 


so  that 


1^ 

K 


K  -JiN. 


2  (0-X  )  +  (K-M  )A 
'•i=l  ^  ^ 


with  the  right  side  converging  to  A  as  K  -»  <».  Thus,  choosing  A  <  P, 

K 

KP  +  2  Xj  >  KXj^  for  K  sufficiently  large.  One  can  thus  apply  part  (c)  of 
i=l 

Theorem  2  of  [3] ,  giving 
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n 

cJ(nP)  =  t  S^log 


‘1+0  ■ 

n  . 

1+X. 

+  2 

nP  +  2"  (X  -0) 

1  +  - — I - 

n(l+e) 


n 


Since  log  - >  0,  -  2  log 

n  "i=l 


1+0 


1+X 


n*' 


n 


0,  Similarly.  —  2  {X.^0)  0. 

"  i=i  ^ 


1  + 


1+0 


:  part  (1)  is  proved,  and 


Thus,  one  again  has  lim  ~  C^(nP)  =  f  log 

n 

this  also  completes  the  proof  of  part  (3). 

If  dim  range(Rj^)  =  M  <  oo,  then  in  the  immediately  preceding  result  one 
has  for  n  sufficiently  large. 


„  M  rM  +  nP  +  2^ 

cJ(nP)  =  t  2^1og 


•  iP. 

J=1  1 


M(l+p.) 


where  ^  ^  the  eigenvalues  of  S.  In  this  case, 

lim  ^(P)  =0,  so  that  R  >  0  is  not  permissible. 

n 


The  theorem  is  now  proved  when  0  <  <».  If  0  =  oo,  then  obviously 
P  1 

=  0.  Part  (2)  of  the  theorem  can  be  ignored,  since 


1  + 


1+0 


C^(P)  >  t  log 

0  =  «>  cannot  occur  unless  range(Rj^)  is  infinite-dimensional.  Thus,  it  only 
remains  to  prove  part  (1),  and  this  is  equivalent  to  showing  that 

n  °  0  =  «>.  If  there  exists  an  integer  M  such  that 

n 


1 


n 


\+l  >  P  +  ~  2  Xi  for  all  n  >  M,  then 
i=l 


M 


n  2n  ^ 

j=l 


rP  +  ^  2^  ..X.  +  1 

M  1=1  1 


n 


n 


1  +  X. 

J 


=  0. 


Suppose  that  there  exists  a  subsequence  (n^)  of  the  integers  such  that 


1  ^ 

for  all  k  >  1,  X  .  -  —  2  X.  <  P.  This  gives 
k  ^  \  i=l  ^ 
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lln,  1  (^(nP)  =  lim  ^  ^  log 
n  k  Tc  1=1 


■p  - 

X  ,  -  —  l\. 
V'  ”k  .1=1 

+  1  +  X  ^,1 

V' 

1  +  X. 

1 

M 


M 


^  2^;, 
k  k  1=1 


Fp  t  1  +  X  ^,1 

P  +  1  +  X 

—  1  \ 

4-  1  ■?  m  -  V  1 

n,  +1 
k 

1  +  X. 

2n,  .  „  , 

1  +  X 

1 

k  k  i=M+l 

i 

for  any  fixed  integer  M.  Now,  since  —  2^  X  ,-X.  <  P,  and 

"k  1=1  L  V'  "J  ■ 


since 


1  Sk  V'  . 


—  ^  1  as  k  -4  00.  we  must  have  that  —  2  —  is  bounded,  so  that 

Tc  i=l  i  ^  i=l 


vi 

then 


<  Cq  for  some  Cq  <  00  and  all  k  >  1.  The  first  term  on  RHS(t)  above  is 


^ 


P  +  1  +  C. 


1  +  X, 


=  0. 


We  now  have ,  f or  any  M  >  1 , 


lln,  i  C5(„P)  i  ll™  ^  log 

n  k  K  i=M+l 


P  +  1  +  X 


+1 


1  +  X. 

1 


<  lim  ^  ^ 

k  i=M+l 


1  ^ 


P  +  X  -  X. 

1 


1  +  X. 
1 


<  lim  ^  2 

k  \  i=M+l 


X  -  X. 

1 


1  +  X 


M+l 


1  +  X 


M+l 


Since  M  is  arbitrary  and  X^  -»  oo,  Hm  i  C^(nP)  =  0,  and  thus  CL(P)  =  0  when 

n  ^ 

9  =  0). 
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Bounds  on  Coding  Capacity  of  the  Discrete-Time  Gaussian  Channel 

We  now  consider  the  following  situation.  A  zero-mean  Gaussian  stochastic 
process  {N^,  t  =  1,2,...}  is  represented  by  a  bounded,  non-negative,  self- 
adjoint  operator  in  R^^  is  an  infinite  matrix  with  Rj^(i,j)  =  EN.N..  The 
constraint  is  given  in  terms  of  a  second  such  operator  R^  in  The  basic 

J-  j. 

assumption  to  be  made  is  that  range(R^)  contains  range(R^). 

A  simple  example  of  such  a  channel  and  constraint  is  the  memoryless 

Gaussian  channel  with  R^  =  I  (leading  to  an  average  power  constraint)  and  R 

2  2 

given  by  Rj^(i,  j)  =  “,-5.  ..  with  a  >  e  for  all  j  >  1,  some  e  >  0. 

In  the  discrete— time  channel,  a  code  (lc,n,e}  is  a  set  of  Ic  code  words 

Xj^,..,Xj^  and  corresponding  decoding  sets  _ satisfying  the  constraints 

given  below,  with  the  requirement  that  each  x^  belong  to  The  decoding  sets 
^te  thus  Borel  sets  in  IR  .  The  constraints  on  the  code  words  are  that 

is  the  n-dimensional  Euclidean 


-t  „2 


IIX-- llm  _  <  nP,  where  11x11^  =  HR  ^  xll^; 

1  W,n  W,n  W,n  n  n 

norm ,  and  R^  ^  is  the  restriction  of  R^  to  {l,2,..,n}x{l,2,..,n}.  As  before , 


we  require  that  pjj{y:  y+x.  €  C.)  >  1  -  e  for  i  <  k,  where  pJJ  is  the  measure 


on 


IB[IR  ]  induced  from  by  the  map  q^:  x  (x^  ,X2.  .  .  ,x^)  .  R  >  0  is  an  admissible 

n.R 

rate  if  there  exists  a  sequence  of  codes  (([e  ^  ],n^,e^  })  with  &  0  as 


n.  ->  00 
1 


\AI 

The  capacity  C^(P)  is  the  supremum  over  the  set  of  admissible  rates. 

An  exact  expression  for  the  coding  capacity  of  the  discrete-time  Gaussian 
channel  is  given  in  [8].  In  some  applications,  the  value  of  the  coding 
capacity  will  be  difficult  to  determine,  as  it  involves  rather  detailed 
knowledge  of  the  spectrum  of  the  operator  S,  defined  above.  In  such  cases  it 
is  useful  to  have  bounds  on  coding  capacity.  For  example,  a  lower  bound 
enables  one  to  strive  toward  communicating  at  a  rate  that  is  certain  to  be 
admissible.  We  give  here  upper  and  lower  bounds  on  coding  capacity. 
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Theorein  2:  Suppose  that  N  is  Gaussian.  Let  6^  be  the  smallest  and  0  the 

i  K 

largest  limit  point  of  the  spectrum  of  the  operator  S.  Then 


log 


1  + 


(i+Gj^).  -  2  ^  +  (i+ep 


If  N  is  not  Gaussian,  and  H™(N)  <  «>,  then 


C^(P)  <  t  log 


1  + 


(1+epj- 


Proof:  The  upper  bound  can  be  obtained  from  part  (1)  of  Theorem  1.  That  is,  we 

can  identify  IR  with  H^,  the  subspace  of  consisting  of  all  elements  x  such 

that  (x)^  =  0  for  i  >  n.  The  constraint  that  any  admissible  code  word  belong 

to  thus  imposes  an  additional  constraint  beyond  those  imposed  in  proving 

00 

the  theorem:  this  gives  C^(P)  i  2  log 


1  + 


(i+epj- 


To  prove  the  lower  bound,  we  can  of  course  assume  that  0..  <  00.  yje  then 

K 

simply  mimic  the  proof  of  part  (3)  of  Theorem  1,  but  now  defining  )L£^  to  be  the 
Gaussian  measure  with  zero  mean  and  covariance  matrix 


= 


M 

Q  ^  n2  n 

2  R;f,u.®R„u. 


X  ~  l+0„+5 

K  1=1 

T1  5  ^ 

where  the  {u^ ,  are  determined  as  follows,  {v^ ,  i  <  M^}  are  o.n.  elements 

Tl  "^2 

in  IR  such  that  ^  1  +  +  5;  such  elements  always  exist  [8]. 

{u^,  i<M^}  are  then  defined  by  u^  =  where  is  the  unitary  operator  in 


\R^  satisfying  R^  =  R^  (i  +s  )2u^. 

^  N,n  W.n*-  n  n''  n 


□ 


JMVA  -  11/12/87  -  12 


References 


I.W.  McKeague,  On  the  capacity  of  channels  with  Gaussian  and  nonGaussian 
noise.  Inform,  and  Control ,  51,  153-173  (1981). 

I.W.  McKeague  and  C.R.  Baker,  The  coding  capacity  of  mismatched  Gaussian 
channels,  IEEE  Trans,  on  Inform.  Theory,  32,  431-436  (1986). 

C.R.  Baker,  Capacity  of  the  mismatched  Gaussian  channel,  IEEE  Trans,  on 
Inform.  Theory  33  (to  appear,  1987). 

C.R.  Baker,  Information  capacity  of  Gaussian  channels,  in  Recent  Advances 
in  Communication  and  Control  Theory,  ed.  by  R.E.  Kalman,  G.I. 
Marchuk,  A.E.  Ruberti,  A.J.  Viterbi;  pp.  2-15,  Optimization 
Software,  Inc.,  New  York,  1987. 

R.G.  Douglas,  On  majorization,  factorization,  and  range  inclusion  of 
operators  on  Hilbert  space,  Proc.  Amer .  Rath.  Soc. ,  17,  413-415 
(1966). 

F.  Riesz  and  B.  Sz-Nagy,  Functional  Analysis,  Ungar,  New  York,  1955. 

C.R.  Baker,  Capacity  of  the  Gaussian  channel  without  feedback.  Inform, 
and  Control,  37,  70-89  (1978). 

C.R.  Baker.  Coding  capacity  of  discrete-time  Gaussian  and  nonGaussian 
channels,  to  appear. 


JMVA  -  11/12/87  -  13 


#;  1 

• 

.  "  .  1 

I 


J- 


4 


« 


